{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "540ff471-dcea-4b3e-9c0c-a3173f1c640e",
   "metadata": {},
   "source": [
    "# Refine"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "158b08a8-32d3-4397-ad37-75870416226b",
   "metadata": {},
   "source": [
    "## Load Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "bbbac556-bb22-47e2-b8bf-80818d241858",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/suo/miniconda3/envs/llama/lib/python3.9/site-packages/deeplake/util/check_latest_version.py:32: UserWarning: A newer version of deeplake (3.6.7) is available. It's recommended that you update to the latest version using `pip install -U deeplake`.\n",
      "  warnings.warn(\n"
     ]
    }
   ],
   "source": [
    "from llama_index import SimpleDirectoryReader"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "bc0d4087-1ee3-4c38-94c0-b34f87ea8aca",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "reader = SimpleDirectoryReader(\n",
    "    input_files=[\"../data/paul_graham/paul_graham_essay.txt\"]\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "7934bb4a-4c0f-4833-842f-7fd47e16eeae",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "docs = reader.load_data()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "bf6b6f5c-5852-41be-8ce8-d94c520e0e50",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "text = docs[0].text"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "efed56ee-fcd3-439c-a1b2-53c643f15c8e",
   "metadata": {},
   "source": [
    "## Summarize"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "4ae4dad4-6044-4c9c-becd-4e2908b54a30",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "from llama_index.llms import OpenAI\n",
    "\n",
    "llm = OpenAI(model=\"gpt-3.5-turbo\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "e65c577c-215e-40e9-8f3f-c23a09af7574",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "from llama_index import ServiceContext\n",
    "\n",
    "service_context = ServiceContext.from_defaults(llm=llm)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "52c48278-f5b2-47bb-a240-6b66a191c6db",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "from llama_index.indices.response import Refine\n",
    "\n",
    "summarizer = Refine(service_context=service_context, verbose=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "834ac725-54ce-4243-bc09-4a50e2590b28",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "> Refine context: have been outnumbered. I was in decent shape at...\n",
      "> Refine context: for stores — in Lisp, of course.\n",
      "\n",
      "We were worki...\n",
      "> Refine context: future. I didn't particularly want to start ano...\n",
      "> Refine context: Summer Founders Program instead of a summer job...\n",
      "> Refine context: to think about other things I could work on. Ho...\n"
     ]
    }
   ],
   "source": [
    "response = summarizer.get_response(\"who is Paul Graham?\", [text])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "a600aa73-74b8-4a20-8f56-1b273417f788",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Paul Graham is an individual who co-founded Y Combinator, a renowned startup accelerator. He played a significant role in the early stages of Y Combinator, including the creation of the Summer Founders Program (SFP). The SFP aimed to fund and support startup teams during the summer, providing them with resources and mentorship. This funding model, based on a combination of previous deals and MIT grad student stipends, proved to be successful and fair.\n",
      "\n",
      "In addition to his work with Y Combinator, Paul Graham is also known for his involvement with Hacker News, a news aggregator and discussion platform. He created Hacker News as a resource for startup founders, but it later evolved into a platform for intellectual curiosity and discussions on various topics.\n",
      "\n",
      "Paul Graham has also written numerous essays on a wide range of subjects, including entrepreneurship, programming, and life experiences. These essays often provide insights and perspectives based on his own experiences and decision-making processes. He has shared his thoughts on topics such as choosing what to work on, the evolution of computers, his time in Florence, painting, and the impact of rapid change on customs and fields.\n",
      "\n",
      "Overall, Paul Graham is recognized as a prominent figure in the startup ecosystem, known for his contributions to Y Combinator, Hacker News, and his thought-provoking essays.\n"
     ]
    }
   ],
   "source": [
    "print(response)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.16"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
